Drama, a connectionist model for robot learning: experiments on grounding communication through imitation in autonomous robots

The present dissertation addresses problems related to robot learning from demonstra¬ tion. It presents the building of a connectionist architecture, which provides the robot with the necessary cognitive and behavioural mechanisms for learning a synthetic lan¬ guage taught by an external teacher agent. This thesis considers three main issues: 1) learning of spatio-temporal invariance in a dynamic noisy environment, 2) symbol grounding of a robot's actions and perceptions, 3) development of a common symbolic representation of the world by heterogeneous agents.We build our approach on the assumption that grounding of symbolic communication creates constraints not only on the cognitive capabilities of the agent but also and especially on its behavioural capacities. Behavioural skills, such as imitation, which allow the agent to co-ordinate its actionn to that of the teacher agent, are required aside to general cognitive abilities of associativity, in order to constrain the agent's attention to making relevant perceptions, onto which it grounds the teacher agent's symbolic expression. In addition, the agent should be provided with the cognitive capacity for extracting spatial and temporal invariance in the continuous flow of its perceptions. Based on this requirement, we develop a connectionist architecture for learning time series. The model is a Dynamical Recurrent Associative Memory Architecture, called DRAMA. It is a fully connected recurrent neural network using Hebbian update rules. Learning is dynamic and unsupervised. The performance of the architecture is analysed theoretically, through numerical simulations and through physical and simulated robotic experiments. Training of the network is computationally fast and inexpensive, which allows its implementation for real time computation and on-line learning in a inexpensive hardware system. Robotic experiments are carried out with different learning tasks involving recognition of spatial and temporal invariance, namely landmark recognition and prediction of perception-action sequence in maze travelling.The architecture is applied to experiments on robot learning by imitation. A learner robot is taught by a teacher agent, a human instructor and another robot, a vocabulary to describe its perceptions and actions. The experiments are based on an imitative strategy, whereby the learner robot reproduces the teacher's actions. While imitating the teacher's movements, the learner robot makes similar proprio and exteroceptions to those of the teacher. The learner robot grounds the teacher's words onto the set of common perceptions they share. We carry out experiments in simulated and physical environments, using different robotic set-ups, increasing gradually the complexity of the task. In a first set of experiments, we study transmission of a vocabulary to designate actions and perception of a robot. Further, we carry out simulation studies, in which we investigate transmission and use of the vocabulary among a group of robotic agents. In a third set of experiments, we investigate learning sequences of the robot's perceptions, while wandering in a physically constrained environment. Finally, we present the implementation of DRAMA in Robota, a doll-like robot, which can imitate the arms and head movements of a human instructor. Through this imitative game, Robota is taught to perform and label dance patterns. Further, Robota is taught a basic language, including a lexicon and syntactical rules for the combination of words of the lexicon, to describe its actions and perception of touch onto its body

Robot Learning from Failed Demonstrations

Robot Learning from Demonstration (RLfD) seeks to enable lay users to encode desired robot behaviors as autonomous controllers. Current work uses a human's demonstration of the target task to initialize the robot's policy, and then improves its performance either through practice (with a known reward function), or additional human interaction. In this article, we focus on the initialization step and consider what can be learned when the humans do not provide successful examples. We develop probabilistic approaches that avoid reproducing observed failures while leveraging the variance across multiple attempts to drive exploration. Our experiments indicate that failure data do contain information that can be used to discover successful means to accomplish tasks. However, in higher dimensions, additional information from the user will most likely be necessary to enable efficient failure-based learnin

Reaching with multi-referential dynamical systems

We study a reaching movement controller for a redundant serial arm manipulator, based on two principles believed to be central to biological motion control: multi-referential control and dynamical system control. The resulting controller is based on two concurrent dynamical systems acting on different, yet redundant variables. The first dynamical system acts on the end-effector location variables and the second one acts on the joint angle variables. Coherence constraints are enforced between those two redundant representations of the movement and can be used to modulate the relative influence of each dynamical system. We illustrate the advantages of such a redundant representation of the movement regarding singularities and joint angle avoidanc

Movement curvature planning through force field internal models

Human motion studies have focused primarily on modeling straight point-to-point reaching movements. However, many goal-directed reaching movements, such as movements directed towards oneself, are not straight but rather follow highly curved trajectories. These movements are particularly interesting to study since they are essential in our everyday life, appear early in development and are routinely used to assess movement deficits following brain lesions. We argue that curved and straight-line reaching movements are generated by a unique neural controller and that the observed curvature of the movement is the result of an active control strategy that follows the geometry of one's body, for instance to avoid trajectories that would hit the body or yield postures close to the joint limits. We present a mathematical model that accounts for such an active control strategy and show that the model reproduces with high accuracy the kinematic features of human data during unconstrained reaching movements directed toward the head. The model consists of a nonlinear dynamical system with a single stable attractor at the target. Embodiment-related task constraints are expressed as a force field that acts on the dynamical system. Finally, we discuss the biological plausibility and neural correlates of the model's parameters and suggest that embodiment should be considered as a main cause for movement trajectory curvatur

Dynamic updating of distributed neural representations using forward models

In this paper, we present a continuous attractor network model that we hypothesize will give some suggestion of the mechanisms underlying several neural processes such as velocity tuning to visual stimulus, sensory discrimination, sensorimotor transformations, motor control, motor imagery, and imitation. All of these processes share the fundamental characteristic of having to deal with the dynamic integration of motor and sensory variables in order to achieve accurate sensory prediction and/or discrimination. Such principles have already been described in the literature by other high-level modeling studies (Decety and Sommerville in Trends Cogn Sci 7:527-533, 2003; Oztop etal. in Neural Netw 19(3):254-271, 2006; Wolpert etal. in Philos Trans R Soc 358:593-602, 2003). With respect to these studies, our work is more concerned with biologically plausible neural dynamics at a population level. Indeed, we show that a relatively simple extension of the classical neural field models can endow these networks with additional dynamic properties for updating their internal representation using external commands. Moreover, an analysis of the interactions between our model and external inputs also shows interesting properties, which we argue are relevant for a better understanding of the neural processes of the brai

Linearization and Identification of Multiple-Attractor Dynamical Systems through Laplacian Eigenmaps

Dynamical Systems (DS) are fundamental to the modeling and understanding time evolving phenomena, and have application in physics, biology and control. As determining an analytical description of the dynamics is often difficult, data-driven approaches are preferred for identifying and controlling nonlinear DS with multiple equilibrium points. Identification of such DS has been treated largely as a supervised learning problem. Instead, we focus on an unsupervised learning scenario where we know neither the number nor the type of dynamics. We propose a Graph-based spectral clustering method that takes advantage of a velocity-augmented kernel to connect data points belonging to the same dynamics, while preserving the natural temporal evolution. We study the eigenvectors and eigenvalues of the Graph Laplacian and show that they form a set of orthogonal embedding spaces, one for each sub-dynamics. We prove that there always exist a set of 2-dimensional embedding spaces in which the sub-dynamics are linear and n-dimensional embedding spaces where they are quasi-linear. We compare the clustering performance of our algorithm to Kernel K-Means, Spectral Clustering and Gaussian Mixtures and show that, even when these algorithms are provided with the correct number of sub-dynamics, they fail to cluster them correctly. We learn a diffeomorphism from the Laplacian embedding space to the original space and show that the Laplacian embedding leads to good reconstruction accuracy and a faster training time through an exponential decaying loss compared to the state-of-the-art diffeomorphism-based approaches.Comment: Paper Accepted at Journal of Machine Learning Research 23 (2022

A dynamical system approach to realtime obstacle avoidance

This paper presents a novel approach to real-time obstacle avoidance based on Dynamical Systems (DS) that ensures impenetrability of multiple convex shaped objects. The proposed method can be applied to perform obstacle avoidance in Cartesian and Joint spaces and using both autonomous and non-autonomous DS-based controllers. Obstacle avoidance proceeds by modulating the original dynamics of the controller. The modulation is parameterizable and allows to determine a safety margin and to increase the robot's reactiveness in the face of uncertainty in the localization of the obstacle. The method is validated in simulation on different types of DS including locally and globally asymptotically stable DS, autonomous and non-autonomous DS, limit cycles, and unstable DS. Further, we verify it in several robot experiments on the 7 degrees of freedom Barrett WAM ar

Iterative Estimation of Rigid-Body Transformations: Application to Robust Object Tracking and Iterative Closest Point

Closed-form solutions are traditionally used in computer vision for estimating rigid body transformations. Here we suggest an iterative solution for estimating rigid body transformations and prove its global convergence. We show that for a number of applications involving repeated estimations of rigid body transformations, an iterative scheme is preferable to a closed-form solution. We illustrate this experimentally on two applications, 3D object tracking and image registration with Iterative Closest Point. Our results show that for those problems using an iterative and continuous estimation process is more robust than using many independent closed-form estimation